Article and Journal-Level Metrics in Massive Research Evaluation Exercises: The Italian Case

Introduction Article level metrics are usually the preferred choice for research evaluation. However, for recent articles they may be integrated or substituted considering some measure of journal impact (Abramo et al., 2012). The use of journal level metrics is also often considered as particularly appealing for administrative purposes, because of their readily availability, easiness to use and comprehensibility (Bordons et al., 2002). On the other hand, the IF is often criticized on the grounds of its possible biases and lack of methodological consistency (Vanclay, 2012). The aim of our paper is to provide evidence about the effects of the use of journal level metrics on the results of a massive research evaluation exercise like the one that has been performed in Italy with reference to the period 2004-2010 (VQR 2004-2010, see Ancaiani et al., 2015). More specifically, in the following we evaluate the effects of the use of the impact factor (IF) on the ranking of Italian Universities at the aggregate level, at the area level and for individual researchers.